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DIMENSIONAL  REDUCTION  FOR  FILTERS  WITH  TIME-SCALE 

SEPARATION 

AFOSR  GRANT  NUMBER  FA9550-08-1-0206 

N.  Sri  Namachchivaya  and  Richard  B.  Sowers 
Department  of  Aerospace  Engineering  and  Department  of  Mathematics 
University  of  Illinois  at  Urbana-Champaign 

Abstract 

This  project  outlines  a  collection  of  problems  which  combine  techniques  of  model 
reduction  and  filtering.  The  basis  of  this  work  is  a  collection  of  limit  theories  for 
stochastic  processes  which  model  dynamical  systems  with  multiple  time  scales. 
Multiple  time  scales  occur  in  many  real  systems,  and  reflect  different  orders  of 
magnitudes  of  rates  of  change  of  different  variables.  These  different  time  scales 
often  allow  one  to  find  effective  behaviors  of  the  fast  time  scales.  In  systems  sub¬ 
ject  to  both  bifurcations  and  noise  (which  form  one  of  the  main  components  of  this 
project),  various  singular  perturbations  problems  must  be  understood.  When  the 
rates  of  change  of  different  variables  differ  by  orders  of  magnitude,  efficient  data 
assimilation  can  be  accomplished  by  constructing  nonlinear  filtering  equations  for 
the  coarse-grained  signal.  We  consider  the  conditional  law  of  a  signal  given  the 
observations  in  a  multi-scale  context.  In  particular,  we  study  how  scaling  inter¬ 
acts  with  filtering  via  stochastic  averaging.  We  combine  our  study  of  stochastic 
dimensional  reduction  and  nonlinear  filtering  to  provide  a  rigorous  framework  for 
identifying  and  simulating  Liters  which  are  specifically  adapted  to  the  complexities 
of  the  underlying  multi-scale  dynamical  system. 

This  is  the  final  report  for  AFOSR  GRANT  NUMBER  FA9550-08- 1-0206, 
which  was  awarded  in  August  2008  and  ended  December  2011.  This  research 
involved  the  work  of  the  PI,  N.  Sri  Namachchivaya,  Co-PI,  Richard  B.  Sowers, 
and  graduate  students  at  University  of  Illinois  at  Urbana-Champaign  (UIUC)  in 
collaboration  with  a  graduate  student  at  the  Humboldt  University,  Berlin,  Ger¬ 
many. 

Introduction 

The  first  objective  of  current  project  is  concerned  with  certain  methods  of  dimen¬ 
sional  reduction  of  nonlinear  systems  with  symmetries  and  small  noise  [1,2].  In 
the  presence  of  a  separation  of  scales,  where  the  noise  is  asymptotically  small, 
one  exploits  symmetries  to  use  recent  mathematical  results  concerning  stochas¬ 
tic  averaging  to  find  an  appropriate  lower-dimensional  description  of  the  system. 
Reduced  models  can  be  used  in  place  of  the  original  complex  models,  either  for 
simulation  and  prediction  or  real-time  control.  To  this  end,  reduced  models  [3,4], 
often  provide  qualitatively  accurate  and  computationally  feasible  descriptions. 

The  second  objective  is  to  derive  a  low-dimensional  filtering  equation,  that 
determines  conditional  law  of  a  plant,  in  a  multi-scale  environment  given  the 
observations.  This  project  is  less  concerned  with  specific  applications  and  more 
focused  on  some  of  the  theoretical  aspects  that  deal  with  reduced  dimensional 
nonlinear  Liters.  In  particular,  we  showed  the  efficient  utilisation  of  the  low¬ 
dimensional  models  of  the  signal  to  develop  a  low-dimensional  Lltering  equation. 


We  combine  two  ingredients,  namely,  stochastic  dimensional  reduction  discussed 
above  and  nonlinear  filtering  [6,7].  We  achieved  this  through  the  framework  of 
homogenisation  theory  which  enables  us  to  average  out  the  effects  of  the  fast 
variables. 

To  introduce  the  basic  idea  of  filtering  in  a  multi-scale  environment,  let  (hi,  P) 
be  a  probability  space.  We  consider  nonlinear  Rm  x  Rn-valued  signal  processes 
( Z£,X£ )  and  an  Revalued  observation  process  Y£  given  by  the  SDE’s 

dZ;  =  e-^Zl  XI) dt  +  e-1/27(Z£t,  X£ )dWt,  Z£  =  V 

(1)  dX£t  =  b(Z£,  XI) dt  +  a{Zl  X£)dVt,  X£  =  £ 
dYt£  =  h{Z£,X£)dt  +  dBu  Y£  =  0 

where  W,  V  and  B  are  independent  Wiener  processes  and  r)  and  £  are  random 
initial  conditions  which  are  independent  of  W,  V  and  B.  Let  £  be  a  small  pa¬ 
rameter  that  measures  the  ratio  of  slow  and  fast  time  scales.  Hence  the  dynamics 
of  (1)  are  separated  into  two  scales,  where  Z£  and  Xs  represent  the  fast  and  slow 
variables,  respectively.  The  generator  of  the  Markov  processes  (Z£ ,  A"£)  is  then 
of  the  form 

(2)  £?e(p  =  +  3fs<p, 

for  all  £  G  (0, 1)  and  all  G  C'°°(Rm+r!),  where  SZp  and  2z?s  represent  generators 
of  fast  and  slow  variables. 

The  main  objective  of  filtering  theory  is  to  estimate  the  statistics  of  the  signal 

X)  =f  (Zt,X]r)  at  time  t  based  on  the  information  in  the  observation  process  Y£ 
up  to  time  t,  that  is,  on  the  basis  of  the  sigma-algebra,  ^V£.  More  precisely  for 
each  t  >  0,  we  want  to  find  the  conditional  law  of  X\  given  that  is,  we  want 
to  compute  P{£f  G  A |^£}  for  all  A  G  ^(Rm+n).  The  insight  of  filtering  theory 
is  that  one  can  construct  this  conditional  measure  via  a  stochastic  PDE  which  is 
a  recursive  equation  driven  by  the  observation  process. 

If  the  above  conditional  measure  admits  a  smooth  density,  p£,x(t7  y)  with  respect 
to  Lebesgue  measure  i.e.,  P{j£f  G  7l|2^e}  =  fAp£,x(t,  y)dy  for  all  A  G  ^(Rm+n), 
then  u£(t,  y),  the  un-normalised  density ,  solves  the  Zakai  equation 

(3)  du£(t,i)  =  AZ*u£{t,i)dt  +  u£(t,f)h(t)dYts,  u£( 0,  •)  =  pt, 

where  is  the  adjoint  operator  of  the  (with  respect  to  Lebesgue  measure  on 
Rm+n)  anq  has  density  pr 

Since  we  are  interested  in  the  slowly-varying  coordinates  which  usually  describe 
the  essential  coarse-grained  dynamics,  the  focus  of  our  work  is  to  estimate  the 
signal  XI  at  time  t  on  the  basis  of  the  sigma- algebra  ^e.  More  precisely  for  each 
t  >  0,  we  want  to  find  the  conditional  law  P{A"te  G  A\^£j  for  all  A  G  tZ(Rn).  For 

each  t  >  0,  define  the  (7([0,  t];  Rd)-valued  random  variable  Yfoti  as  (Yf0  A(w)  =f 
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Yf  (co)  for  all  co  £  Q  and  s  £  [0,  t].  The  point  of  filtering  is  that  for  each  t  >  0  and 
£  G  [0, 1),  there  is  a  measurable  map 

(4)  ir£t  :  C([0,t\-,M.d)  -£  ^>(Rn) 

such  that  for  each  t  >  0  and  A  £  (nf (YJ0 t]))(A)  =  P{Yt£  £  A\&t£}. 

Informally,  our  focus  is  in  the  behavior  of  the  filters  when  system  (1)  can  be 
averaged  with  respect  to  fxx(dz),  the  unique  invariant  measure  of  the  fast  variable 
Zf,  i.e.,  AApHx  =  0.  Notice  that  the  dimension  of  the  signal  (the  coarse-grained 
dynamics)  is  reduced  to  n.  Letting  u°(t,x )  satisfy  the  Zakai  equation  for  the 
reduced  system 

du°(t,  x )  =  J?sii0(t,  x)dt  +  u°(t,  x)h(x)dYt° , 

one  might  hope  that  u£  converges  to  u°.  ( u°  does  not  depend  on  fast  variable  z.) 
As  in  (4),  there  is  a  map 

7Tt°  :  <?([(),  — >■  ^(Mn) 

such  that  (7rt°(Y°  ,))(A)  =  P{Yt°  £  A\^Vt0}.  However,  we  want  to  use  the  original 
observation  Y£  instead  of  Y°  to  get  a  filter  for  the  reduced  system  since  Y£  is  the 
actual  data  we  would  obtain.  Our  main  claim  [9]  is 

Theorem  0.1.  For  each  t  >  0, 

Jim  E  dnm  « ( ■ ■ ,  Yf0)t] ) ,  tt°  ( ■ ■ ,  Yf0)t] ) )  =  0 . 

Here  d^(R)  is  the  standard  Prohorov  metric  on  and  the  statement  is  con¬ 

vergence  in  probability.  In  other  words,  if  you  are  only  interested  in  finding  the 
conditional  distribution  of  the  slow  variables,  you  might  as  well  use  the  averaged 
dynamics. 

Homogenized  Hybrid  Particle  Filter  (HHPF): 

Particle  methods  have  a  long  history;  we  refer  the  reviewer  to  [7]  for  an  extensive 
treatment  of  this  area.  The  basic  idea  is  to  replace  the  expected  values  of  the 
quantities  in  the  statement  of  nonlinear  Liters,  by  suitable  Monte-Carlo  sample 
averages.  The  basic  requirements  for  a  particle  method  are  the  simulation  of 
independent  samples  of  the  signal,  called  particles,  with  the  same  stochastic  law 
as  the  signal,  and  the  re-sampling  of  these  particles  to  incorporate  the  information 
from  the  observations. 

Our  approach  described  here,  not  only  reduces  the  computational  burden  for 
real  time  applications  but  also  helps  solve  the  problem  of  particle  degeneracy.  In 
the  case  where  the  signal  solves  a  stochastic  differential  equation,  the  simulation 
of  each  particle’s  SDE  is  usually  implemented  with  discrete  time  Euler  or  Miltsein 
approximations.  The  conditional  distribution  of  the  signal  at  the  current  time  t  is 
approximated  by  the  locations  of  the  particles  and  their  weights,  specifically.  The 
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set  of  particles  are  then  adjusted  in  some  manner  to  conform  to  each  observation 
by  either  assigning  each  particle  a  scalar  weight  value,  w"(£)  that  is  calculated  for 
the  jth  particle  at  (xf3 ,  z^’3)  to  account  for  the  information  from  the  observation 
up  to  time  t  by  interchanging  the  state  values  associated  with  each  particle  in 
some  manner  or  by  cautious  novel  branching  techniques  presented  here. 

An  Illustrated  Example: 

In  this  section,  we  provide  some  evidence  that  suggests  that  the  proposed  ap¬ 
proaches  are  likely  to  work,  with  sufficient  accuracy,  in  practice.  We  apply  the 
HHPF  to  an  example  to  illustrate  its  potential  for  high-dimensional  complex  prob¬ 
lems. 

Consider  the  following  signal  model: 

Z£t  =  --{Z£-X£)+^Wt,  Z£  =  Zo,  xet  =  -(Zt£)3+sin(7Tt)+cos(v/2vrf),  X£  =  x0 
with  the  observation 

y;  =  l  W)2 + Bt. 


Figure  f.  Signal  and  Observation  (left)  and  HMM  solution  and 
the  original  Signal  (right) 


FIGURE  2.  Full  branching  particle  filter  (left)  and  HHPF  (right) 
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Figure  1  (left)  shows  a  typical  plot  for  this  multiscale  system  with  slow  and 
fast  signals,  Xte  and  Z£,  respectively,  and  with  the  observations  Yt£.  We  then 
applied  the  algorithms  described  above  to  the  this  problem  using  MATLAB.  Fig¬ 
ure  1  (right)  compares  the  HMM  solution  with  the  analytical  solution.  The  results 
from  the  branching  particle  filter  and  the  HHPF  are  given  in  Figures  2  (left)  and 
2  (right),  respectively.  The  time  taken  for  these  simulations  are  448  sec.  and  15 
sec.  respectively  with  a  2  GHZ  Intel  Core  2  Duo  MacBook.  The  parameters  we 
have  used  are:  N  =  200,  M  =  2,  rir  =  10,  Nm  =  200,  St  —  le  —  4,  At  =  5e  —  2, 
£  =  le  — 3.  Here  rir  is  the  number  of  micro  time  steps  skipped  to  ignore  transients 
and  Nm  is  the  number  of  micro  time  steps.  These  preliminary  results  are  critical 
in  order  to  demonstrate  that  the  proposed  methods  show  promise. 

The  theoretical  aspect  of  data  assimilation  was  accomplished  by  constructing 
nonlinear  filter  equations  for  the  coarse-grained  signal  (i.e.,  the  lower  dimensional 
Zakai  equation),  as  explained  in  [8,  9].  We  showed  that  a  multiscale  particle 
method  could  be  effectively  applied  to  large  dimensional  systems.  The  separation 
of  time-scales  facilitated  the  construction  of  the  particle  filter  which  allows  the 
dimensional  reduction  of  the  system  needed  to  solve  for  each  particle. 

In  summary,  this  novel  computational  method  combined  homogenization  of  ran¬ 
dom  dynamical  systems,  the  results  of  [8,  9]  on  reduced  order  nonlinear  filtering, 
with  sequential  Monte  Carlo  methods  to  reduce  the  effective  number  of  variables 
in  the  evaluation  of  the  conditional  distribution  needed  in  the  Bayesian  filter  for 
data  assimilation.  For  computational  purposes,  the  conditional  distribution  n  is 
approximated  by  particle  filters;  the  system  of  particles  is  resampled  in  the  ex¬ 
treme  cases  described  in  the  previous  paragraph.  It  is  worthwhile  to  note  that  the 
convergence  results  of  both  stochastic  averaging  and  the  branching  particle  filter 
are  obtained  through  the  method  of  the  martingale  problem.  Thus,  it  is  natural 
to  consider  the  convergence  result  of  HHPF  in  the  framework  of  the  martingale 
problem,  which  is  left  for  future  work. 
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